Artificial Neural Network Architecture Research Articles

A Neural Network Forecasting Approach for the Smart Grid Demand Response Management Problem

Demand response management (DRM) plays a crucial role in the prospective development of smart grids. The precise estimation of electricity demand for individual houses is vital for optimizing the operation and planning of the power system. Accurate forecasting of the required components holds significance as it can substantially impact the final cost, mitigate risks, and support informed decision-making. In this paper, a forecasting approach employing neural networks for smart grid demand-side management is proposed. The study explores various enhanced artificial neural network (ANN) architectures for forecasting smart grid consumption. The performance of the ANN approach in predicting energy demands is evaluated through a comparison with three statistical models: a time series model, an auto-regressive model, and a hybrid model. Experimental results demonstrate the ability of the proposed neural network framework to deliver accurate and reliable energy demand forecasts.

Spintronic Artificial Synapses Using Voltage‐Controlled Multilevel Magnetic States

AbstractNeuromorphic computing offers energy‐efficient computations for large‐scale data processing compared to conventional von Neumann computing. The artificial synapse, a key element for learning and memory operations in neuromorphic computing, requires multi‐state characteristics and the capability to change and store its states. The implementation of hardware‐based artificial synapses using nonvolatile memory provides significant advantages in terms of energy consumption and circuit area compared to their currently employed CMOS‐based counterparts. In this regard, spintronic devices have emerged as a promising candidate due to their desirable properties for artificial synapses, including multilevel formability, non‐volatility, and outstanding writing performance. In this study, spintronic artificial synapses utilizing voltage‐controlled multilevel magnetic states and energy‐ and area‐efficient artificial neural network architectures associated with them are demonstrated. The multilevel states are created by gradually modulating the magnetic easy‐axis orientation from perpendicular to in‐plane and vice versa, which is achieved either by sequentially applying gate voltage pulses or by adjusting the pulse width of the gate voltage. Based on these spintronic artificial synapses, convolutional neural network (CNN) and spiking neural network (SNN) architectures are constructed, demonstrating high recognition accuracy for the MNIST dataset with improved energy efficiency and a reduced circuit area.

A Neural Network Approach for Software Reliability Prediction

The increasing reliance on computer software has raised significant concerns regarding software reliability evaluation. Over the past four decades, various software reliability models have been developed, encompassing both parametric and nonparametric approaches. However, no single model has demonstrated effectiveness in handling all types of datasets. In response to this challenge, the deep neural network (DNN), a powerful deep learning model, has emerged as a promising solution. By leveraging the flexibility and adaptability of artificial neural networks (ANN), the DNN model exhibits remarkable prediction performance by capturing training variables and exploring deeper layers. This study presents an approach that utilizes a DNN model based on an ANN architecture with multiple activation functions. This approach aims to estimate software reliability and predict the frequency of software flaws. Through extensive experimental analysis and validation, the proposed DNN model surpasses the predictive accuracy achieved by existing parametric and nonparametric models. These results highlight the potential of the DNN model in effectively addressing the complexities involved in software reliability evaluation. By combining the power of deep learning with the adaptability of ANN, the proposed model offers a promising and accurate solution for software reliability prediction.

Deep-Learning-Based Strong Ground Motion Signal Prediction in Real Time

Processing ground motion signals at early stages can be advantageous for issuing public warnings, deploying first-responder teams, and other time-sensitive measures. Multiple Deep Learning (DL) models are presented herein, which can predict triaxial ground motion accelerations upon processing the first-arriving 0.5 s of recorded acceleration measurements. Principal Component Analysis (PCA) and the K-means clustering algorithm were utilized to cluster 17,602 accelerograms into 3 clusters using their metadata. The accelerograms were divided into 1 million input–output pairs for training, 100,000 for validation, and 420,000 for testing. Several non-overlapping forecast horizons were explored (1, 10, 50, 100, and 200 points). Various architectures of Artificial Neural Networks (ANNs) were trained and tested, such as Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM) networks, and CNN-LSTMs. The utilized training methodology applied different aspects of supervised and unsupervised learning. The LSTM model demonstrated superior performance in terms of short-term prediction. A prediction horizon of 10 timesteps in the future with a Root Mean Squared Error (RMSE) value of 8.43 × 10−6 g was achieved. In other words, the LSTM model exhibited a performance improvement of 95% compared to the baseline benchmark, i.e., ANN. It is worth noting that all the considered models exhibited acceptable real-time performance (0.01 s) when running in testing mode. The CNN model demonstrated the fastest computational performance among all models. It predicts ground accelerations under 0.5 ms on an Intel Core i9-10900X CPU (10 cores). The models allow for the implementation of real-time structural control responses via intelligent seismic protection systems (e.g., magneto-rheological (MR) dampers).

Dynamically reconfigurable all-optical neural network based on a hybrid graphene metasurface array

In recent years, optical neural networks (ONNs) have received considerable attention for their intrinsic parallelism and low energy consumption, making them a vital area of research. However, the current passive diffractive ONNs lack dynamic tunability after fabrication for specific tasks. Here, we propose a dynamically reconfigurable diffractive deep neural network based on a hybrid graphene metasurface array, wherein the transmission and refractive index of each pixel can be finely adjusted via gate voltage. This capability enables the tailored modulation of the incident light’s amplitude and phase at each pixel, aligning with specific task requirements. The simulation results show the attainability of a dynamic modulation range of 7.97dB (ranging from −8.56dB to −0.591dB). Additionally, this proposed diffractive neural network platform incorporates an ultrathin structure comprising a one-atom-thick graphene layer and nanoscale metallic metastructures, rendering it compatible with complementary metal-oxide-semiconductor technology. Notably, a classification accuracy of 92.14% for a single-layer neural network operating in the terahertz spectrum is achieved based on the calculation result. This proposed platform presents compelling prospects for constructing various artificial neural network architectures with applications ranging from drug screening to automotive driving and vision sensing.

Environmental Lighting towards Growth Effect Monitoring System of Plant Factory using ANN

Malaysia is currently driven to become another most developed country in the world. Among other priority sector is Food Sustainability. Along the process, our vegetable supply-demand keeps increasing by year. Compared to traditional systems, closed systems or its other name called hydroponic is getting more important for plant production, with artificial light which has many potential advantages, including better quality transplants, shorter production time and less resource use. To gain full profit from it, the quality of vegetables needs to be controlled efficiently. Climate conditions, especially temperature and light intensity, have a significant impact on vegetable growth and yield, as well as nutritional quality. Plant growth and development are influenced by a variety of environmental factors, the most important one is light intensity. Among the problems to be tackled in this research are plant growth manual observation, light intensity variation and abundance of growth-related data to be evaluated manually. Therefore, to solve these problems, the specific type of vegetable used here is lettuce. The proposed methods are, observation of plant growth conducted automatically round the clock in intervals of 15 minutes for the whole month (estimated mature period of lettuce), using images captured. At the same time, the proposed light intensity which is red & white to the ratio of 2:1 (optimum ratio recommended by previous researchers) will be used. The issue of data to be evaluated manually will be solved using Artificial Neural Network (ANN) architecture, in specific Deep Learning. Concisely, the results & analysis shows the research is successfully developed for plant growth monitoring by using artificial neural network which, reached 80% to 90% accuracy in the training and validation session that made the architecture sufficient for determining the growth of the said vegetable. This is indeed foreseen, will highly assist the farmer in better monitoring the growth rate of the plant.

Exam Assessor Tool: An Automated System for Efficient Answer Sheet Evaluation

With Education 4.0 and four quadrant approach number of innovations have gone into academics for efficient, experiential, and outcome-based education however assessment schemes are still very much dependent on manual assessment methods which are time-consuming and cumbersome. The grading system can sometimes be irrational, with diversified schemes for the same course and can also be biased. Covid 19 pandemic caused a global economic avalanche like we’ve never experienced in our lifetime. Many countries have implemented control measures such as blockades and curfews. The education system in this chaos saw a silver lining with academics shifting to online mode, with paradigm shift in teaching, assessment techniques too need to evolve. Work done is an effort to ease the process of assessment, a machine learning assisted model is developed that automates subjective answer evaluation in the education sector. Our project involved several crucial steps, including grayscale conversion, Natural Language Processing (NLP) for data cleansing, data splitting, and training an artificial neural network (ANN) to predict scores based on extracted features. ANN-based system grades subjective responses without human intervention, reducing the workload of teachers and professors. Model constructed an ANN architecture with three layers using Rectified Linear Activation Unit (ReLU) and Sigmoid activation functions. Trained model was incorporated into a user-friendly web application using the Streamlit library. Model design gives a major boost in grading efficiency and accuracy while providing valuable feedback to students. Research surveys were conducted, and a dataset was constructed for training and testing the model. study yielded an accuracy of 83.14% after employing techniques such as text cleaning, preprocessing, and feature extraction.

Assessment of mechanical and physicochemical properties of palm fiber composites: Effect of alkaline treatment and volume alterations

This study assesses the impact of alkaline treatments and volume fractions on biocomposites composed of a high-density polyethylene (HDPE) matrix reinforced with date palm tree fibers (FPDS). Tensile tests were conducted on both untreated and NaOH-treated biocomposites. Additionally, fiber analysis was performed using scanning electron microscopy (SEM) and X-ray diffraction (XRD). The results reveal higher strength and stiffness compared to HDPE, albeit with limited plasticity making the material brittle. The NaOH treatment enhances certain mechanical properties. Further assessments encompassed hardness, density, melt index, and Izod impact tests. Two volume fractions, 20% and 25%, of FPDS were tested. The study establishes a correlation between empirical predictions and artificial neural network (ANN) models. Notably, an ANN architecture consisting of two input factors, 10 hidden nodes, and one output provides the analysis of mechanical properties. This investigation highlights the potential of FPDS-reinforced HDPE biocomposites, emphasizing their mechanical performance under various treatments and fiber levels.

Prediction of the discharge capacity of piano key weirs using artificial neural networks

Abstract The discharge capacity of the piano key weir (PKW) is an important flow feature which ultimately decides the type and geometric design of PKWs. In the present research work, the different architecture of artificial neural networks (ANNs) was employed to predict the discharge capacity of the trapezoidal piano key weir (TPKW) by varying geometric parameters (Si/So, Wi/Wo, Bi/Bo, L/W and α). Furthermore, adaptive neuro-fuzzy interference system (ANFIS), support vector machines (SVMs) and non-linear regression (RM) techniques were also applied to compare the performance of best ANN models. The performance of each model was evaluated using statistical indices including scatter index (SI); coefficient of determination (R2), and mean square error (MSE). The prediction capability of all the models was found to be satisfactory. However, results predicted by ANN-22(H-15) [R2 = 0.998, MSE = 0.0024, SI = 0.0177] was more accurate than ANFIS (R2 = 0.995, MSE = 0.00039, SI = 0.0256), SVM (R2 = 0.982, MSE = 0.000864, SI = 0.0395) and RM (R2 = 0.978, MSE = 0.001, SI = 0.0411). It was observed that Si/So, Wi/Wo and L/W geometric ratios have the greatest effect on the discharge performance of TPKW. Furthermore, sensitivity analysis confirmed that h/P is the most influencing ratio which may considerably affect the discharge efficiency of the TPKW. It was found that ANN models having a single hidden layer and keeping neurons three times of input parameters in hidden layers generated better results.

Predicting asphaltene precipitation in CO2 flooding processes using artificial intelligence techniques

The removal of asphaltene depositions from crude oil wellbores and tubing is a very difficult and expensive process. Therefore, a very accurate predictor for the Asphaltene Precipitation (AP) is a very important tool for decision making. In this paper, the AP during the injection of carbon dioxide into crude oil reservoirs is forecasted using an artificial neural network (ANN). For this purpose, experimental data is collected from six different oil fields. 70% of the collected data is utilized for training and to develop the proposed ANN. Trainlm training algorithm was found to be the best ANN architecture for AP forecasting. In order to validate the capability of the designed ANN 30% of the unseen data is utilized for validation. The forecasting results show the Mean Square Error (MSE) of the forecasting is 0.0018 that confirms the high accuracy of the model. In order to evaluate the capability of the proposed methodology, the results of proposed ANN are compared with modified Hirschberg model. Comparison confirmed the proposed ANN has better capability and higher accuracy. In the final stage, the effects of various operating parameters on the AP during gas injection are examined. The results show that the most sensitive parameter is the reservoir temperature. In addition, there is a high correlation between an increase in the AP with an increase in carbon dioxide concentration in liquid phase.

DEEP LEARNING FOR SPACE DEBRIS REMOVAL

The advances in deep learning have revolutionized the field of artificial intelligence. These advances, as well as new tasks and requirements in space exploration, have led to an increased interest in these deep learning methods among space scientists and practitioners. The problems of controlling the attitude and relative motion of spacecraft are considered for both traditional and new missions, such as contactless space debris removal. Both supervised and reinforcement learning is used to solve such problems based on various architectures of artificial neural networks, including convolutional ones. The possibility of using deep learning together with methods of control theory is analyzed to solve the considered problems more efficiently. The difficulties that limit the application of these methods for space applications are highlighted. The necessary research directions for solving these problems are indicated.

Sampling complex topology structures for spiking neural networks.

Spiking Neural Networks (SNNs) have been considered a potential competitor to Artificial Neural Networks (ANNs) due to their high biological plausibility and energy efficiency. However, the architecture design of SNN has not been well studied. Previous studies either use ANN architectures or directly search for SNN architectures under a highly constrained search space. In this paper, we aim to introduce much more complex connection topologies to SNNs to further exploit the potential of SNN architectures. To this end, we propose the topology-aware search space, which is the first search space that enables a more diverse and flexible design for both the spatial and temporal topology of the SNN architecture. Then, to efficiently obtain architecture from our search space, we propose the spatio-temporal topology sampling (STTS) algorithm. By leveraging the benefits of random sampling, STTS can yield powerful architecture without the need for an exhaustive search process, making it significantly more efficient than alternative search strategies. Extensive experiments on CIFAR-10, CIFAR-100, and ImageNet demonstrate the effectiveness of our method. Notably, we obtain 70.79% top-1 accuracy on ImageNet with only 4 time steps, 1.79% higher than the second best model. Our code is available under https://github.com/stiger1000/Random-Sampling-SNN.

Predicting total biogas potential of food waste using the initial output of biogas potential tests as input data to train an artificial neural network

Quantification of biogas potential is important for predicting anaerobic digestion operability and price. This study uses data from 446 biogas potential tests to train and test a multilayer perceptron artificial neural network (ANN) to forecast total biogas production using the evolution at the start of the experiment (3–14 days) as input data. ANN architecture (training algorithm, activation function, hidden nodes, regularisation, and input data) was optimised using response surface methodology. Best conditions (accuracy/computational speed) were obtained using adaptive moment estimation (adam) training algorithm and rectified linear unit (ReLU) activation function. When using three days of biogas production data, the accuracy of the model was reasonable (r2test = 0.881, r2validation = 0.879), although this increased significantly for 7 days (r2test = 0.953, r2validation = 0.925), or 14 days (r2test = 0.971, r2validation = 0.953). The highest accuracy was reported for readily digestible substrates (sugars and carbohydrates) and macronutrient mixtures. The methodology could be used to shorten prediction times of biogas potential tests.

Green synthesis of carbon nanotubes functionalized with iron nanoparticles and coffee husk biomass for efficient removal of losartan and diclofenac: Adsorption kinetics and ANN modeling studies

The presence of emerging contaminants in wastewater poses a global environmental challenge, requiring the development of innovative materials or methods for their treatment. This study focused on the production of green functionalized carbon nanotubes (CNTs) and using them in the adsorption of the pharmaceuticals Losartan (LOS) and Diclofenac (DIC). The efficiency of the methodology was verified by characterization techniques. Elemental composition analysis indicated a significant increase in the iron content after the green functionalization, proving the effectiveness of the method. Thermogravimetric analysis showed similar thermal degradation profiles for pristine CNTs and functionalized CNTs, indicating better post-functionalization thermal stability. BET analysis revealed mesoporous characteristics of CNTs, with increased surface area and pore volumes after functionalization. X-Ray diffraction confirmed the preservation of the lattice structure of the CNTs post-functionalization and post-adsorption, with changes in peak broadening suggesting surface modifications. LOS and DIC adsorption were evaluated via kinetic studies at four different concentrations (0.1–0.4 mmol/L) that were best represented by the pseudo-second order model, suggesting chemisorption mechanisms, with faster and higher uptakes for DIC (0.084–0.261 mmol/g; teq = 5 min) when compared to LOS (0.058–0.235 mmol/g; teq = 20 min). The curves were also studied via artificial neural networks (ANN) and revealed that the best ANN architecture for representing the experimental data is a network with [3 5 5 2] neurons trained using the Bayesian-Regularization algorithm and the Log-sigmoid (hidden layers) and Linear (output layer) transfer functions. The desorption study showed that CaCl2 had better performance in CNT regeneration, reaching its removal capacity above 50% up to 3 cycles, for both pharmaceuticals. These findings reveal the potential of the developed material as a promising adsorbent for targeted removal of pollutants, contributing to advances in the remediation of emerging contaminants and the application of artificial intelligence in adsorption research.

Artificial Neural Network Prediction Model for Maternal Health Services Quality in Nigeria

This study presents an innovative approach in predicting the quality of maternal health services in Nigeria using artificial neural networks (ANNs). It focuses on a binary classification task in categorizing maternal health services facilities into "High_Quality" and "Low_Quality" classes. Dataset was collected from diverse healthcare facilities in Nigeria through a WHO Health Resources and Services Availability Monitoring System (HeRAMS) cross sectional-survey report. The collected data was preprocessed; the ANN architecture was developed and trained using back-propagation method. The network was modeled using feed-forward modeling approach with 16 variables at the input layer, 2 hidden layers of 65 and 55 neurons each, and an output of 2 neurons.  Hence, 87.17% accuracy with only 2.48% deviation at the 50 epoch was achieved. The model's performance was evaluated using standard metrics of accuracy (87%), precision (88%), recall (89%), and F1-score (88%) to assess its ability to accurately classify maternal health services facilities. The findings of the study can have significant implications to policymakers, healthcare providers, and maternal health advocates to optimize healthcare resources and achieve better maternal health outcomes in Nigeria. Data availability, potential biases in the data, and the subjective nature of some quality-related indicators were acknowledged as limitations. However, despite these challenges, the ANN classification model remains a valuable tool for supporting decision-making in maternal healthcare facility delivery service in Nigeria. Artificial neural network, Binary classification, Health services, Maternal health, Predictive modeling

A novel CFD-ANN approach for plunger valve optimization: Cost-effective performance enhancement

This paper introduces a novel computational fluid dynamics-artificial neural network (CFD-ANN) approach that has been devised to enhance the efficiency of plunger valves. The primary emphasis of this research is to achieve an optimal equilibrium between hydraulic flow and geometric configuration. This study is a novel contribution to the field as it explores the flow dynamics of plunger valves using Computational Fluid Dynamics (CFD) and proposes a unique methodology by incorporating Machine Learning (ML) for performance forecasting. An artificial neural network (ANN) architecture was developed using a thorough comprehension of flow physics and the impact of geometric parameters acquired through computational fluid dynamics (CFD). Using optimization, the primary aspects of the Artificial Neural Network (ANN), including the learning algorithm and the number of hidden layers, have been modified. This refinement has resulted in the development of an architecture exhibiting a remarkably high R2 value of 0.987. This architectural design was employed to optimize the plunger valve. By utilizing Artificial Neural Networks (ANN), a comprehensive analysis comprising 1000 distinct configurations was effectively performed, resulting in a significant reduction in time expenditure compared to relying on Computational Fluid Dynamics (CFD). The result was a refined arrangement that achieved maximum head loss, subsequently verified using computational fluid dynamics (CFD) simulations, resulting in a minimal discrepancy of 2.66%. The efficacy of artificial neural networks (ANN) becomes apparent due to their notable cost-efficiency, along with their capacity to produce outcomes that are arduous and expensive to get through conventional optimization research utilizing computational fluid dynamics (CFD).

High performance artificial visual perception and recognition with a plasmon-enhanced 2D material neural network

The development of neuromorphic visual systems has recently gained momentum due to their potential in areas such as autonomous vehicles and robotics. However, current machine visual systems based on silicon technology usually contain photosensor arrays, format conversion, memory and processing modules. As a result, the redundant data shuttling between each unit, resulting in large latency and high-power consumption, seriously limits the performance of neuromorphic vision chips. Here, we demonstrate an artificial neural network (ANN) architecture based on an integrated 2D MoS2/Ag nanograting phototransistor array, which can simultaneously sense, pre-process and recognize optical images without latency. The pre-processing function of the device under photoelectric synergy ensures considerable improvement of efficiency and accuracy of subsequent image recognition. The comprehensive performance of the proof-of-concept device demonstrates great potential for machine vision applications in terms of large dynamic range (180 dB), high speed (500 ns) and low energy consumption per spike (2.4 × 10−17 J).

Artificial neural networks ensemble methodology to predict significant wave height

The forecast of wave variables are important for several applications that depend on a better description of the ocean state. Due to the chaotic behaviour of the differential equations which model this problem, a well know strategy to overcome the difficulties is basically to run several simulations, by for instance, varying the initial condition, and averaging the result of each of these, creating an ensemble. Moreover, in the last few years, considering the amount of available data and the computational power increase, machine learning algorithms have been applied as surrogate to traditional numerical models, yielding comparative or better results. In this work, we present a methodology to create an ensemble of different artificial neural networks architectures, namely, MLP, RNN, LSTM, CNN and a hybrid CNN–LSTM, which aims to predict significant wave height on six different locations in the Brazilian coast. The networks are trained using NOAA’s numerical reforecast data and target the residual between observational data and the numerical model output. A new strategy to create the training and target datasets is demonstrated. Results show that our framework is capable of producing high efficient forecast, with an average accuracy of 80%, that can achieve up to 88% in the best case scenario, which means 5% reduction in error metrics if compared to NOAA’s numerical model, and a increasingly reduction of computational cost.

EPHA: An Energy-efficient Parallel Hybrid Architecture for ANNs and SNNs

Artificial neural networks (ANNs) and spiking neural networks (SNNs) are two general approaches to achieve artificial intelligence (AI). The former have been widely used in academia and industry fields; the latter, SNNs, are more similar to biological neural networks and can realize ultra-low power consumption, thus have received widespread research attention. However, due to their fundamental differences in computation formula and information coding, the two methods often require different and incompatible platforms. Alongside the development of AI, a general platform that can support both ANNs and SNNs is necessary. Moreover, there are some similarities between ANNs and SNNs, which leaves room to deploy different networks on the same architecture. However, there is little related research on this topic. Accordingly, this article presents an energy-efficient, scalable, and non-Von Neumann architecture (EPHA) for ANNs and SNNs. Our study combines device-, circuit-, architecture-, and algorithm-level innovations to achieve a parallel architecture with ultra-low power consumption. We use the compensated ferrimagnet to act as both synapses and neurons to store weights and perform dot-product operations, respectively. Moreover, we propose a novel computing flow to reduce the operations across multiple crossbar arrays, which enables our design to conduct large and complex tasks. On a suite of ANN and SNN workloads, the EPHA is 1.6× more power-efficient than a state-of-the-art design, NEBULA, in the ANN mode. In the SNN mode, our design is 4 orders of magnitude more than the Loihi in power efficiency.

Predictive Modeling of Fracture Behavior in Ti6Al4V Alloys Manufactured by SLM Process

This study focuses on ductile fracture behavior prediction for Ti6Al4V alloys fabricated via Selective Laser Melting (SLM). A modified Gurson-Tvergaard-Needleman (GTN) model characterizes void growth and shear mechanisms under uniaxial stress. The research explores the impact of Artificial Neural Network (ANN) architecture, specifically hidden layers and neurons, on predicting fracture parameters. Results reveal that increasing hidden layers substantially enhances accuracy, particularly for fracture displacement. Notably, predicting maximum force requires fewer layers than fracture displacement. Using selected layers and neurons, the system consistently achieved R2-values exceeding 0.99 for both maximum force and fracture displacement. The study identifies the initial void volume fraction (f0) parameter as having the most significant influence on both properties.